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Abstract 

We consider a distributed non cooperative control setting in which systems are 
interconnected via state constraints. Each of these systems is governed by an 
agent which is responsible for exchanging information with its neighbours and 
computing a feedback law using a nonlinear model predictive controller to avoid 
violations of constraints. For this setting we present an algorithm which generates 
a parallelizable hierarchy among the systems. Moreover, we show both feasibility 
and stability of the closed loop using only abstract properties of this algorithm. 
To this end, we utilize a trajectory based stability result which we extend to the 
distributed setting. 
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1. Introduction 

Distributed control problems can arise either naturally, i.e. by a set of coupled 
systems which shall be controlled, see, e.g., Dold and Stursberg [4], or if a large 
problem is decomposed into smaller, again coupled problems, see Rawlings and 



Mayne 11171. Chapter 10] or Scattolini [|20n for an overview. In the latter case, the 



general idea is that smaller problems are solvable easier and faster which allows 
to even overcompensate the computational effort to coordinate these systems, cf. 



Richards and How 11191. Section 7]. In either case, one distinguishes between coop- 



erative control which features a centralized objective, and non cooperative control 
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where the objectives of the systems are independent from each other. Using a cen- 
tralized objective there are several possibilities to divide the optimization problem 
into subproblems. If suitable conditions hold then similar performance of the dis- 
tributed control obtained from these subproblems and of the centralized control 
can be shown, see, e.g., Rawlings and Mayne tl7j, Chapter 10] or Giselsson and 
Rantzer fl. 

Throughout this work we focus on the non cooperative control setting of sys- 
tems driven by independent dynamics and control objectives, but coupled by con- 
straints. For each system we impose an agent which exchanges state information 
with its neighbours and uses its local objective to compute a local control which 
satisfies the coupling constraints. For the computing task we focus on feedback 
design via nonlinear model predictive controller (NMPC) which minimizes the 
distance of the current state to the desired equilibrium over a finite time horizon. 
To show asymptotic stability of an NMPC closed loop, one often imposes addi- 
tional stabilizing terminal constraints and costs, see, e.g., Keerthi and Gilbert lfl5ll 
or Chen and Allgower H respectively. Since such terminal constraints may re- 
quire long optimization horizons, we focus on the plain NMPC setting without 
those modifications. In the non distributed case, stability for such problems has 
been shown in Grime et al. Ill ill whereas the distributed case is treated in Grime 
and Worthmann lfl2ll using the algorithm of Richards and How [ill, 19ll - 

Here, we first prove a stability idea outlined in Grime and Worthmann Ill2ll 
using the trajectory based setting of Grime and Pannek Chapter 7]. This proof 
allows us to reduce the horizon length in the distributed case while maintaining 
suboptimality estimates and stability like behavior of the closed loop. Secondly, 
since the computing time of the NMPC control law for each agent is not negligible, 
we present an algorithm which allows us to execute these computations in parallel 
using priority and deordering rules as well as a decision memory. From Rawlings 
and Mayne M 1 VL Chapter 10] it is known that for the non cooperative control setting 
one can only expect to reach a Nash equilibrium. Although such a solution may 
be far from the optimal centralized solution, the closed loop solutions may still 
be stable and maintain the coupling constraints. For the proposed algorithm we 
present conditions under which feasibility of the closed loop is guaranteed and 
present necessary as well as sufficient conditions for asymptotic stability using 
only abstract properties of both the priority and the deordering rule. While here 
we focus on the plain NMPC case, we also outline how feasibility and stability 
results can be obtained using NMPC with terminal constraints or cost. 

The paper is organized as follows: First, in Section |2]we formally define the 
problem under consideration for which we show different stability results for the 
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distributed case in Section [3] In the central Section @] we present a covering 
algorithm which allows us to generate a hierarchy of agents and run the computa- 
tions of each hierarchy level in parallel. Using this algorithm, we show necessary 
and sufficient conditions for feasibility and stability of the resulting closed loop 
and also how much parallelism can be achieved. Instead of a separated exam- 
ple section, we use an analytical example throughout the entire work to present 
the improvement of the stability result but also to illustrate the abstract functions 
used within the proposed algorithm in Section |4] Finally, we draw conclusions in 
Section \5\ and present ideas for future research based on the presented work. 

2. Setup and Preliminaries 

Throughout this work we consider a set of nonlinear discrete time systems 

x p {n+ 1) = fp(x p (n),u p (n)), p e V : = {1, . . . , P}, n e N (1) 

with x p (ri) e X p and u p (n) e U p and No denoting the set of natural numbers 
including zero. Here, X p and U p , p e V, are assumed to be arbitrary metric 
spaces denoting the state space and the set of admissible control values of the p-th 
system, respectively. The metrics to measure distances between two elements of 
X p or of Up are denoted by d Xp : X p x X p — > IR> and d Up : U p x U p — > M>o 
where R>o denotes the positive reals including zero. In the following we denote 
the solution of a system p of (Q~|) corresponding to the initial value x p (0) = x° p and 
the control sequence u p (k) e U p , k = 0, 1,2, . . ., by x u p {k, xp). 

In order to define our goal we say that a continuous function a : M> — » R>o 
is of class 'Koa if it satisfies a(0) = 0, is strictly increasing and unbounded. A 
continuous function y : M> —> M> is called a class function if it satisfies 
y(0) = 0, is strictly increasing in each component and is unbounded. A continuous 
function /3 : R> xR>o —> M>o is of class %C if it is strictly decreasing in its second 
argument with lim Moo )S(r, = for each r > and satisfies /?(•, t) e ^K^ for each 
t > 0. Moreover, !B r (x) denotes the open ball with center x and radius r and for 
arbitrary x\,x 2 e X we denote the distance from x\ to x 2 by ||xi||. V2 = d x {x\, x 2 ). 

For the set of systems CQ> the overall system is given by 

x(n + 1) = f(x(n), u(n)), neN (2) 

with state x(n) = (xi(n) T , . . . , xp(n) T ) T e X = X\ x . . . x X P and control u{n) = 
(ui(n) T , up{n) T ) T e U = U\ x . . . x Up. Now, our goal is to asymptotically 
stabilize system © at a desired equilibrium point x ret e X, i.e. to fulfill the 
following: 
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Definition 1. Let x"* g X be an equilibrium for a system ©, i.e., there exists 
u e U such that /(x ref , u) = x ref . Then we say that x Tef is locally asymptotically 
stable if for a given control sequence (u(n)) ne n there exist r > and a function 
jS e 7C£ such that the inequality 

\\x u (n,x°)\\ x ^<mA\^,n) (3) 

holds for all x° e S r (x vef ) and all neN . 

Additionally, the solution x ll (-,x°) shall satisfy state and control constraints. 
Throughout this work, we incorporate such constraints by considering suitable 
subsets of the overall state and control value space X c X, U c U for system ©. 
As a result, systems (Q3 are coupled via the constraint sets X and U although the 
respective dynamics are decoupled. The following example illustrates this setting 
and will be used throughout this paper. 

Example 2. Consider two cars attempting to cross a one lane brigde, i.e. one car 
has to wait, cf. Figure [T]for an illustration. Suppose the discrete time dynamics of 
the cars are given by 

x p (n + 1) = x p (ri) + u p (n) 

with u p e Up = {-1,0, l} 2 and x p e Z 2 for p = 1,2. Since the cars shall not 
collide, we obtain the restriction 

(*i,i,*u) T * <>2,i,*2,2) T - (4) 
The one lane bridge additionally imposes the constraints 

x p ,2 = if x p ,\ = for p = 1,2 and (5) 

Xl,2 + "1,2 / \ *2,2 / \ *1,2 / \ *2,2 + "2,2 / 

which together form the set X. Hence, the local set of admissible moves may 
depend on which car is allowed to drive first, cf . Figures [la] and [TbJ Note that 
© is the only local constraint whereas — if they are considered - © induces an 
algebraic and © a neighbouring dynamic dependent coupling. 

The purpose of this work is to show conditions under which stability of the 
overall systems can be guaranteed by using only local controllers. To this end, we 
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(a) Admissible moves of X2 if(b) Admissible moves of X2 if (c) Admissible solution 
x\ is off the bridge x\ enters bridge 



Figure 1: Admissible moves for initial conditions x x = (1, 0) T , x; = 0) T 

impose an agent for each system p e V to compute a suitable control sequence 
u p (-) 6 U p ° := {u p (k) | u p (k) e U p Vk e N } and to exchange information with 
other agents in order to verify that the constraints X c X, U c U are satisfied. 
Throughout this work each agent computes its control sequence via a nonlinear 
model predictive controller, a methodology which will be explained after Defini- 
tion|6l below. In order to achieve asymptotic stability of the overall system © we 
develop a covering algorithm to coordinate all agents. The idea of this algorithm 
is the following: Since some subsystems impose constraints on each other, a prior- 
ity rule is used to generate a hierarchy among the subsystems. As a result, agents 
which are on the same hierarchy level can compute their local optimal control 
in parallel while the hierarchy levels remain in serial. Additionally, a deordering 
rule is introduced to repeatedly verify if the hierarchy can be flattened, i.e. if more 
agents can work in parallel. For details on these rules we refer to Section 0] 

Since we want to compute local controls u p we must define the local con- 
straints for each single system f p , p e P. To this end, we "project" the constraint 
set X to the state space of a subset of systems. 

Definition 3. For an index set I p = {pi, . . . , p m } cP with m e N, m < P and p t + 

Pj for all i, j e { 1 , . . . , m\ the set of partial states is defined as Xj = X pi x . . . xX Pm 
and we denote elements of Xj p by Xj = (x Pi , x Pm ). Accordingly, the partial 
state constraint set is defined by 

Xj p := {xj p G X Ip | there is x, e X with x Pi = x for i = I,..., m). 

In case of Example[2l Definition [3] basically means that only those constraints 
induced by the neighbours contained in I p have to be considered, i.e. if agent 



p = I ignores agent p = 2, then only constraint © has to be fulfilled. In the 
general case, this task can be accomplished by using information on neighbouring 
systems which are available to an agent and allow to generate a hierarchy among 
the agents. Here, we assume that this information can be exchanged repeatedly in 
between two time instants n and n + 1 . 

Throughout this work we consider changing network topologies, i.e. the sets 
of neighbours at time instants n and n+ 1 may differ, see Figure[2]for an illustration. 




Time instant n Time instant n + 1 



Figure 2: Possible sequence of communication graphs at time instants n and n + 1 

Additionally, we allow the case that even if neighbouring information of a sys- 
tem q 6 V \ \p\ is known to an agent peP, agent p ignores that information if his 
hierarchy level is higher than the level of the neighbour. Consequently, the depen- 
dency graph which results from this hierarchy may differ from the communication 
graph as illustrated in Figure |3] 




Figure 3: Possible difference of communication and dependency graph 



Moreover, as sent information may be delayed or even lost, we want to allow 
for considering old information on neighbours and variable lengths of this infor- 
mation. Note that the latter may also allow agents to skip recomputations of con- 
trols. Introducing N q as the length and n q as the time instant at which neighbour 
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q has computed the state sequence (x q "(Q), . . . , x q 9 (N q )), we define the exchanged 
neighbouring information as follows: 

Definition 4. Suppose that at time instant n e N agent p knows the state se- 
quences x" q (-) = (x q q (0), x n q q {N q )), N q G N , computed at time instant n q < n 
for a given neighbouring index set I p (n), that is q G I p (n) with p g I p (n). We 
define the neighbouring information as 

I p {n) = {(q,n q ,N q ,x n q 9 (-)) \ q G I p {n)\ 

being an element of the set I p = 2 Q with Q = (P\ {p}) xN x N x X N . 

Knowing the states of neighbouring systems for a certain time period, we can 
define the index set used within the "projection" of the constraint set X. 

Definition 5. For a given time instant n e N and an agent p G P with neigh- 
bouring information I p (n), we call the set of systems q e X„(n) \ {p} which are 
imposing constraints on system p at time instant n + k e N , k > neighbouring 
prediction index set. This set is given by 

I p (n, k) = {qe I p (n) \{p}\n + k<n q + N q }. 

Similar to possible moves of the cars in Example |2] we can use the partial 
state constraint set connected to neighbouring information available to an agent 
and define the set of admissible controls from which the control sequence u p {-) 
can be chosen, cf. Figures [Tal [Tbl 

Definition 6. Given a time instant n e No and an agent p e P with initial value 
x° p and neighbouring information I p {n), we define the set of admissible control 
sequences for system p at time instant n as 

Uf(n, x° p , I p (n)) = {u p (-) g U®° \ for all k = 0, 1, ... we have u p (k) G U p and 

(x u p (k, x° p ), (x q '(k + n- n q )) IpM) ) G X [p]UIpM) }. 

Using an NMPC algorithm is one possibility to compute a control from the 
set of admissible controls. In particular, the method tries to approximate a control 
sequence such that the functional 

oo 

j;(x° p , u p ) = Yj 4), u p (k)) (7) 

k=Q 
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is minimized over all admissible control sequences, that is sequences u p (-) with 
u p (k) = «*(0) for all k e N with u* e Uf(k, x u p (k, x° p ), I p (k)). Here, the function 
t p is a stage cost function penalizing both the distance of the state to the desired 
equilibrium and the used control. A popular choice for this function is £ p (x p , u p ) = 
\\xpWjies + /t||wp|| M ref with weighting parameter A > 0. 

Computing a control minimizing © is, in general, computational intractable. 
To circumvent this issue the NMPC algorithm uses the truncated cost functional 

f/(x p , u p ) = J] tp(x p (k, 4), u P {k)) (8) 

k=Q 

with finite horizon of length N p and initial value jc°. Hence, a finite minimizing 
control sequence u* p e V p '" ad (n, x° p , I p (n)) is computed with 

U p p ' a (n, x° p , I p (n)) = {u p {-) G U p " | for all k = 0, . . . , N p we have u p (k) e V p and 

(x u p (k,x° p ),(x n J(k + 

n ~ n q))l p (n,k)) G X(p|uj- p („ 4)}. 

In the following we assume that a minimizing control sequence exists and denote 
the corresponding optimal value function by 

Vp"(x p (ri), I p (n)) = jnin J p "(x p (n),u p ) 

u p eV/' (n,x p (n),I p (n)) 

where the minimizing control sequence is given by 

u* p = argmin J p "(x p (n),u p ). 

u p eV^, pM (n,x p (n),I p {n)) 

Here, the argmin operator is used in the following sense: given a map a : U — > R, 
a nonempty subset U Q U and a value w* e {/ we write w* = argmin ue ^ a(i<) if and 
only if a{u*) = min ue ^ a(u) holds. Note that we do not require uniqueness of the 
minimizer u*. In case of uniqueness the argmin operator can be understood as an 
assignment, otherwise it is just a convenient way of writing "m* minimizes a(u)'\ 
Having obtained a minimizing sequence only the first element w*(0) of 
the control sequence is implemented. Then the entire problem is shifted forward in 
time by one time instant and both a new initial value and neighbouring information 
need to be obtained. Applying this method iteratively results in a feedback law 
which assigns the first element of the minimizing control sequence «*(•) to the 
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current state of the p-th system x p (n) and the neighbouring information I p (n) of 
the corresponding agent, i.e. a map 



Hp" :( X? (fl),/ ? (fl))H«;(0). (9) 

Accordingly, the closed loop solution of the p-th system is given by 

x p (n+l) = f(x p (n),4"(x P (n),I p (n))) with x p (0) = x° p . (10) 

Using this setting, we first show conditions which guarantee asymptotic sta- 
bility of the closed loop for local controllers. 

3. Stability 

While commonly endpoint constraints or a Lyapunov function type endpoint 
weight are used to ensure stability of the closed loop, see, e.g., the articles of 
Keerthi and Gilbert [150, Chen and Allgower Q3D , Jadbabaie and Hauser IU4I1 and 



Graichen and Kugi [7J], we consider the plain NMPC version without these modi- 
fications. In order to guarantee stability in this case, we use the "relaxed" version 
of the dynamic programming principle, cf. Lincoln and Rantzer Ill6ll . In partic- 
ular, one can show asymptotic stability of © in a trajectory based setting using 



a relaxed Lyapunov condition, see Griine and Pannek fllOl Proposition 7.6]. Note 
that this stability result requires a centralized setting and the horizons to satisfy 
N p = N for all p e V. Hence, for the overall system © we denote the combined 
stage costs by £(x(n), u(n)), the finite and infinite cost functional by J N (x°, u), 
J°°(x°, u) and the corresponding combined value functions by V N (x(n)), V°°(x(ri)) 
which allows us to apply the stability result of Griine and Pannek [10, Proposition 
7.6]: 

Proposition 7. Consider a feedback law fi N : X — > U and the closed loop trajec- 
tory x(-) of © with control u = fi N and initial values x(0) e X to be given. If the 
optimal value function V N : X —* M>o satisfies 

V N (x(n)) > V N (f(x(n),fi N (x(n))) + a€(x(n),fi N (x(n))) (11) 
for some a e (0, 1 ] and all n e No, then 

aV°°(x(n)) < ar(x{n),n N ) < V N (x(n)) < V°°(x(n)) (12) 
holds for all n e N . 
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If, in addition, there exist a,\,ai, a-^ e 7C tx , such that 

Qfi(||x|| x ref) < V N (x) < a2(\\x\\^f) and £{x,u) > aT,{\\x\\ x ^f) (13) 

holds for all x(n) e X with n e No, then there exists a function fi e '1QL which only 
depends on a\,a2, &3 and a such that the inequality 

IW«)IU <)8(IW0)|U,n) (14) 

holds for all n e N , i.e., x behaves like a trajectory of an asymptotically stable 
system. 

The key assumption in Proposition [7J is the relaxed Lyapunov-inequality (fTTT) 
in which a can be interpreted as a lower bound for the rate of convergence. From 
the literature, it is well-known that this condition is satisfied for sufficiently long 
horizons N, cf. Jadbabaie and Hauser Ill4ll . Grimm et al. [18(] or Alamir and 
Bornard [1], and that a suitable N may be computed via methods described in 
Griine and Pannek [fToL Chapter 7] or Giselsson 

Now we consider a distributed setting of Proposition |7J using compositions to 
combine of the set of systems (Q~|). The idea of such compositions is to introduce 
a weighting among the subsystems which in our further analysis will allow for 
increases of costs along the closed loop for some subsystems. 

Proposition 8. Consider feedback laws p p : X p x I p — » V p and closed loop 
trajectories x p {-) of (fTOl) with initial values x p (0) e X p to be given. If the optimal 
value functions V p : X p — > M>o satisfy 

V N p (x p (n)) > V N p {f p (x p {n),ix N p {x p (n), I p (n)))) + ai p (x p (n),p N p (x p (n), I p (n))) (15) 

for some ct £ (0, 1] and all n e No, then for any weighting function y : M p — > M>o, 
y{x) = y T x, y t e E>o we have that (fT2l) holds for all n e No with 

V N (x) :=y({V^(xy\...,V N P {x p )) T ) and £(x,u) := y((A(xi, u y ), . . . , £ P (x P , u P )) T ). 

If, in addition, for every p e V there exist a\, a v 2 , e 'K ca such that 

a\{\\x p \\^i) <V p {x) <a p 2 {\\x p \\^{) and € p (x p , u p ) > o^(\\x p \\^i) (16) 

holds for all x p (n) e X with n e No, then there exists a function fi e 'KL which 
only depends on y, a and all a\, a p 2 , a P , p e P, such that (TT4l) holds for all n e Nq. 
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Proof. Defining the abbreviations V™(x(n)) := (Vf (xi(n)), . . . , V"(x P (n))) T and 
tp(x(n),fi N (x(n),Ip(n))) := . . . J P {x P {n),ii N p (x P (n), I P (n)))) 

we combine all inequalities (fT51) for p e V and obtain 

y(V£(x(n))) > y(Vp(x(n + 1))) + ar r (^(x(n),^(x(n), I P {n)))). (17) 

using linearity of y. Now we use the definition of V N and ^ which gives us (PTTT) . 
Hence, (fT2l) follows directly from Proposition [7] Similarly, (PT41) follows by defi- 
nition of V N and I which together with «,-(>) := y{{a){r), . . . , af (r)) T ), z = 1, 2, 3, 
and again Proposition [7] shows the assertion. □ 

Certainly, condition (fT5l ) would be desireable since it guarantees a decrease in 
Vp for each p e V. In practice, however, one would usually expect Vp to decrease 
for some p e V while it increases for others as shown in the following example: 

Example 9. Consider the setting of Example |2] where we suppose that each agent 
p = 1,2 is optimizing using its running costs £ p (x p ,u p ) = \\x p - J^fWl with 
x r f = (-2, 0) T and x v f = (2, 0) T and complete neighbouring information for ini- 
tial conditions x° { = (1, 0) T , x° 2 = (-1, 0) T . 

Due to the constraints dU)-© one car has to wait before entering the bridge, cf. 
Figure [Tel and even has to move aside as shown in Figure |4a] Without loss of 
generality we assume that system p = 2 moves aside. Although the optimal con- 




-2-1012 -2-1012 -2-1012 



(a) Optimal solution for n = 1(b) Optimal solution for n — (c) Global optimal solution 
and N = 1 and N = 2 



Figure 4: Optimal open and closed loop trajectories 

trol for agent p = 2 for any Af > 2 is identical to the global optimal control, cf. 
Figures gS and gc] we obtain V^(x 2 (0), 7 2 (0)) < V*(jc 2 (1), / 2 (1)) for N = 2 and 
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N = 3. Hence, we cannot guarantee (TT3T ) to hold for these N although the closed 
loop is stable. For larger values of N, however, we obtain 



37 if iV = 4 

V?(x 2 (0),I 2 (0))= 41 UN = 5 and € 2 (x 2 (0),^(x 2 (0), / 2 (0))) = 9, 

[42 if N > 6 

V 2 N (x 2 (l),I 2 (l)) = \ll lf ^ T = 4 c and £ 2 (x 2 (l),$(x 2 (l),I 2 (l))) = 9, 

33 if N > 5 



(v?(x 2 (n), h(n))) n=2 5 N>4 = (24, 10, 5, 1) and 
(Uxi{n),i4{x 2 {n)J 2 {n)))) n=i ^ = (10,9,4, 1) 

and V?(x 2 (n),I 2 (n)) = £ 2 (x 2 (n),fj,%(x 2 (n),I 2 (n))) = if iV > 4 and n >6. Accord- 
ingly, the largest values a such that (fT5T ) holds are a = 5/9 if /V = 4, a = 8/9 
if Af = 5 and a = 1 if /V > 6. Hence, we can use Proposition [8] to conclude 
asymptotic stability if N > 4. 

In an indeep analysis, Grime and Worthmann [fl2[ Theorem 5.3] have shown 
conditions such that for the serial case using the algorithm of Richards and How 



[|18L ll9ll inequalities (IT3T ) and (TT6l hold. Note that although the setting within the 
articles of Richards and How lll8[ [l9ll is for one based on linear dynamics and 
secondly explicitely includes perturbations in the models, this algorithm can also 
be used in a nonlinear setting, cf. Griine and Worthmann [12, Proposition 3.2]. 

Taking a closer look at the proof of Proposition[8]we see that in fact conditions 
(TT5T ) are only required to guarantee (fTTT i to hold. Note that while condition (fT3T ) 
requires a decrease in V p for each p e P, in (fPTI) it suffices that V p , p e Pis 
decreasing under a map y. Moreover, we only require y e 'K^ in the remainder 
of the proof of Proposition [8] to guarantee a, e TCo, i = 1,2,3. Accordingly, we 
obtain the following more general result: 

Proposition 10. Consider feedback laws fx p : X p x I p — » U p an d closed loop 
trajectories x p (-) of (fTOl wzf/z initial values x p (0) e X p to be given. If the optimal 
value functions V p : X p — > M> satisfy §F7§ for some a e (0, 1], 7 e 7C£ and all 
n e No, (fT2l) holds for all neNo wzY/z and i defined as in Proposition^ 

If, in addition, for every p e P there exist a p ,a p 2 ,a P e f K oa such that (fT6l) /zoMs 
/or a// x p (n) e X wzY/z n e No, zTiere existo a function J3 e 7C£ which only 
depends on y, a and all a p ,a p 2 , a P , p €p, such that (PT41) holds for all n e Nq. 
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Proof. Follows directly from the proof of Proposition [8] 



□ 



The conclusion that can be drawn from Proposition ITOl is that the weighting 
function y may allow us to partially violate condition (TT~5T > . Since (fT5T ) is typically 
fulfilled if the horizon N p is large enough, a good choice of y may reduce the 
horizon length N p as we will see in the following example: 

Example 11. Consider Example |9] and suppose y to be the 1-norm, then we ob- 
tain for N = 2 



( 2 



P =i 

2_j i p {x p {n),p N p {x p {n), I p {n))) 



= (31,24, 20,13,5,1) 



n=0,...,5 
\ 



Vp=i 



= (18,13,11,9,4,1) 



'n=0,...,5 



and Zj=i V^x p (n),I p (n)) = Zj=i e p (x p (n),^(x p (n),I p (n))) = Oforn > 6. Hence, 
(TT7T ) holds with a = 4/13 and we obtain asymptotic stability of the closed loop 
by Proposition fTU} Since a > holds for all N > 2 this example illustrates the 
advantage of considering condition (ITVT) instead of (IT3T ). 

As outlined before Proposition [101 under certain conditions the algorithm of 
Richards and How [|18L ll9ll can be applied to generate solutions such that (PT51) and 
(TT6b hold. However, the nature of this algorithm is serial, that is while one agent 
p e V is computing its control, all other agents q e V \ {p} have to wait until 
agent p finished computing. Hence, if the number of systems P is large, such an 
algorithm may cause rather long waiting times, a feature which may be unwanted 
if fast sampling is used. Still, as noted in Richards and How II19l Section 7], due 
to its decentralized nature the dimension of each problem is significantly smaller 
and hence the algorithm reduces the numerical effort compared to a centralized 
solution considerably. 

Apart from the serial nature, the algorithm of Richards and How requires acces- 
sibility to the full neighbouring information, i.e. a full communication graph. 
Additionally, an agent p e P always uses the latest available neighbouring in- 
formation to compute a minimizing control u* p which results in a full dependency 
graph. While the latter condition on the dependency graph may be relaxed easily, 
it is a complex task to obtain a parallel algorithm and to relax the requirement of 
a full communication graph. 
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4. The covering algorithm 



In this section we provide a covering algorithm which is a modification of the 



Algorithm of Richards and How 11181 1 191 1 and allows us to run the agents p e P 



in parallel if they are independent from one another. Unfortunately, working in a 
parallel distributed setting omits the use of standard techniques from optimization 
such as first and second order information of the cost functional and the constraints 
for the interlink between systems to search for optimal controls. 

To circumvent this deficiency we introduce abstract maps IT, : 2 9 — > 2 P 
which denote priority and deordering rules, see Haupt [[Till and Backstrom [0]. 
The aim of this section is to show how much parallelism can be expected using the 
algorithm we propose next and the basic properties of n, being a permutation 
and a self concatenation mapping respectively. 

The structural layout of the algorithm we present now is closely related to the 
NMPC algorithm outlined in Section [2J 

Algorithm 12. Set lists P 1 := (1, . . . , P) and P p := for p = 2, . . , , P, n := and 
I p {n) := 0for/7 = 

1 . Obtain new measurements x p (n) for p e P. 
2a. (Decision memory and deordering rule) For i from 2 to P do 
For j from 1 to %Pi do 

(i) SetI Pj (n):=G(I Pj (n))CI Pj (n) 

(ii) If I Pj (n) = 0, then remove pj from Pj and set P\ := {P\,Pj) 
Else: If in = vam ke p m PkeIp ^ l) m < i holds, then remove pj from Pi 
and set !P ffi := (Pm,Pj) 

2b. Compute a control «*(■) minimizing © or © with x° p = x p (n) and send 
information to all agents q e {q e P \ q e Pj, p e P t and j > i} for p 6 P in 
parallel 

2c. (Priority rule) For i from 1 to P do 

(i) If e {0,1}, goto Step 3. 

Else: Sort index list by setting P t := Il(P ; ) 

(ii) For j from 2 to $P t do 

If system pj violates constraints imposed by systems p k , k < j, 
then set P M := (P i+U j) and I» := J/n) U e P,- \ | 
p^., k < j, induces constraints violated by system pj} and setP , : = 

(iii) Compute a control «*(-, minimizing © or © for all p e Pi+\ in 
parallel and send information to all agents qe{qeP\qe Pj, j > i] 
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3. Implement p. p p (x p (ri),I p (n)) := w*(0), set n := n + 1 and goto Step 1. 

The general idea of the algorithm is to first generate priority lists P , of the systems 
according to the rule II and according to their interconnection with other systems, 
cf. Step 2c, - just as the right-before-left rule in street traffic or the search direc- 
tion in optimization methods. Secondly, these lists are used to remember earlier 
decisions which avoids generating periodic behaviour. This part of the algorithm, 
contained in Step 2a, is inspired by Bland's rule and the lexicographic ordering 
method used in the simplex algorithm to cope with degeneracy. Last, the deorder- 
ing rule which is used together with the memory in Step 2a offers a possibility 
to break up earlier decisions. Proceeding this way avoids blockages and reduces 
both the number of priority lists and thereby the numerical effort to compute the 
control sequences. 

We like to mention that Algorithm [12] can be extended to an iterative com- 
putation of the controls u* , p e P. To this end only a few steps within the opti- 
mization method used to solve the problems of Steps 2b and 2c(iii) are performed. 
Additionally a second loop containing Steps 2b and 2c is introduced which is ter- 
minated if some stopping criterion like the suboptimality based criterion given 
in Grime and Pannek [|9|] is satisfied. Note that the algorithm also allows us to 
stop agents during such an iterative computation, i.e. if (TT3T ) is satisfied for some 
a > a e (0, 1). Since we allowed for using old and even outdated information in 
DefmitionlH the algorithm even allows to block any computations of some agents 
for a certain period depending on the length of an agents prediction without com- 
promising feasibility. 

Given Algorithm[[2l we first consider the question whether a feasible feedback 
\x p p can be computed via Algorithm [T2l 

Theorem 13. Assume a feasible initial value xq e X for system (© to be given. 
Suppose that for all p e P and all n e No we have that the sets of admissible 
controls U a p (n, x p (n), I p (n)) in case of cost functional © or V p p ' ad (n, x p (n), I p (n)) 
in case of cost functional © in Steps 2b and 2c(iii) are not empty, then the closed 
loop solutions (PTOl) satisfy x{n) = (xi(n) T , . . . , xp(n) T ) T e X. 

Proof Using x e X and Vf(0, x p (0), I p (0)) * for all p e P in case of cost 

functional © or Up" (0, x p (0), I p (0)) * for all p e P in case of cost functional 
©, we obtain from Steps 2b and 2c(iii) that optimal controls u* p (-,I p (0)) exist for 
all pep. Hence, by definition of the closed loop in (flOl) and Step 3 we obtain 
that x(l) = (xi(l) T , . . . ,x P (l) T ) T e X holds. Applying the same argumentation 
inductively for all n e No the assertion follows. □ 
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Before showing results for Algorithm [12] together with general priority and 
deordering rules n, 0, we like to illustrate both rules using the example outlined 
in Section [3] The idea of the priority rule is straight forward. In fact, we have 
already used it in Example [9]to solve the blockage in the very first step: 

Example 14. Again consider Example [9j Due to the constraint sets X, U and 
the dynamics of the systems f p , one of the agents p has to move aside first to let 
the system of the other agent pass by before it can proceed towards its desired 
equilibrium. Putting priority of agent p = 1 into a mathematical form, we see that 
II can be implemented as a lexicographic ordering, that is a list Si is mapped to 
its minimal permutation with respect to the dictionary ordering < d induced by the 

total orderings {<i < m } where m is the length of the list X and <,-,/= 1, . . . m 

is the usual ordering < of the natural numbers M. 

Apart from the lexicographic ordering, also other heuristics like the greedy 
heuristic might be used. It is not clear how the priority rule should be chosen in a 
nonlinear setting, and throughout this work we will not focus on this question but 
instead concentrate on general properties of Algorithm [TZl 
The idea of the deordering rule is more involved as it may interfere with the 
idea of keeping track of earlier decisions. The purpose of this rule is to reduce the 
number of the priority lists since Step 2c of Algorithm [T2l is a serial call for all 
lists Pi. Accordingly, agents p e P M always have to wait until all agents p e P t 
have finished computing, a fact we wish to avoid. Note that this serial nature is 
independent from the parallel computation of control sequences w* , p e P ,-. Using 
the deordering rule allows us to "test" whether a system p e P t still interferes 
with all systems p e Pk, k < i, or if it can be inserted into a different priority list 
Pk, k < i, causing the number of lists and hence the number of non parallel steps to 
shrink. Yet even if system p cannot be inserted in a different priority list, applying 
the deordering rule might still result in reducing the size of the neighbouring index 
set I p (n). If this is the case, then the number of constraints of system p is reduced 
which in turn reduces the numerical effort to compute the control sequence u* p . 

Example 15. Consider once more Example [9] with Q(P) = 0. Applying Algo- 
rithm [T2] we obtain that w* depends on the solution of system p = 1 for n e {0, 1} 
only whereas for all n > 2 both problems can be solved in parallel. 

Turning towards the central point of this section, we now analyze how much 
parallelism is possible even if we do not know the exact sorting and testing op- 
erators n, 0. Based on conditions on the priority lists P t our first result shows in 
which case all agents can compute their controls independently from each other: 
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Lemma 16. Suppose that for given systems ©, maps II, : 2 9 — > 2 P and neNo 
we have that P 2 = & holds in Step 2c(i) ofAlgorithm\T2\ Then every agent p e P 
can compute its control sequence independently of all other agents q e P \ {p}. 

Proof. Since P 2 = Step 2c(ii) of Algorithm [12] guarantees that there are no 
systems puP2 e Pi, Pi Pi, such that p x induces a constraint on p 2 which is 
violated by p 2 , i.e. I p (n) = for all p e P. Hence, for each agent p e P the set 
of admissible controls simplifies to 

Uf(n, x° p , I p in)) = {u p (-) e \ u p {k) e U p and x u p {k, x° p ) e X p for all k e N } 

if cost functional © or 

Vp" M (n, x° p , I p (n)) = {u p (-) g Up" \ u p (k) e U p and x u p (k, x° p ) e X p 

fora\\ke{0,...,Np}} 

if cost functional © is considered with x p = x p (n) showing the assertion. □ 

Using the self-concatenation property of the map 0, we can also show that 
under certain conditions the priority lists show dependency of agents: 

Lemma 17. Consider systems CQ), P > 2 to be given. Suppose that applying 
AlgorithmU~2[for given maps II, : 2 r — > 2 r we have that Pi ■£ with i > 2 holds 
for some n >n and all n e F% Then for each system p e P t there exists at least 
one system q e Pj, j < i such that q e I p (n). Moreover, in case cost functional 
© is used, we have 

u* p = argmin j;(x p (n), u p ) t W p d (n, x p (n), I p (n)) C W p d (n, x p (n), 0) 

u p eVf{n,x p (n)3) 

and in case of cost functional © we have 

u p = argmin f p \x p (n), u p ) <£ U p d (n, x p (n), I p (n)) C V N p pM {n, x p (n), 0). 

u p eu" , "'" 1 (n,x I ,(n)$) 

Proof. Suppose that P t + with i > 2 holds for some n >n and all n e No and 
fix p 6 Pi arbitrarily. Suppose furthermore that there exists no q e Pj, j < i such 
that q 6 I p (n) holds. Then, by the deordering rule and Step 2a(i) we obtain that 
there exists n e No such that P t = for all n > n contradicting our assumption. 
Hence, since p 6 P { was chosen arbitrarily, we obtain that for each p 6 P t there 
exists a system q e Pj, j < i such that q e I p (n) holds. 

Now, due to Step 2c(ii) and the fact that there exists a system q e I p in) imposing 
constraints on system p which are violated if q i I p (n) the assertion for both cost 
functionals © and ® follows. □ 
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Now we can use Lemma [FT] to answer the question under which conditions 
asymptotic stability can be shown. In particular, we first prove a necessary condi- 
tion for asymptotic stability of ©. 

Theorem 18. Consider systems (0Q), P > 2 to be given. Suppose that applying 
Algorithm \12\for all maps IT, : 2 9 — > 2 P we have that P t ± 0, i > 2 holds for 
some n >n and all n e No with I p {n, 1) ± (bfor some p e Pj. Then there exists no 
function fi e %£ such that (|T4]) holds for all n e Nq. 



Proof. Fix maps II, : 2 P — » 2 P . Then Lemma \T7\ states that for each system 
p 6 Pi there exists a system q e Pj, j < i such that q e I p (ri). If for any p e P 
and any n e N we have that Vf(n, x p (n), I p (n)) = or V p " M (n, x p (n\ I p (n)) = 
in case if cost functional © or © are used, respectively, we are done since no 
admissible solution exists. Otherwise, we obtain u* p l (-) ± u* p 2 (-) with 

u* p \-)= argmin J p (x p (n),u p ), u* p 2 (-) = argmin J p (x p (n),u p ) 

u p eVf(n,x p (n),I p (n)) ' u p &jf{n,x p (n)j&) 

in case of cost functional © and with 

u* p {-)= argmin J p "(x p (n),u p ), u* p 2 (-) = argmin J p "(x p (n),u p ) 

u p eV>" p *\n,x p (n\I p (n)) u p eV N p " M (n,x„(n),<D) 

in case of cost functional ((8]). 

Hence, due to the fact that x p (k, x{n)) for some k violates a constraint imposed 
by system q which is not violated by x p (k, x(n)), we obtain that the open loop 
trajectories x p (•, x{n)) and x p (-,x(n)) differ. Using I p (n, 1) ^ 0, we can con- 

* 1 « 2 

elude that there exists a 6\ > such that d x (x p (l,x p (n)),x p (l,x p (n))) > 6\ 
holds. Since we always implement the first element of each optimal admissible 
control, we have that d x (f P (x p (n), u* p l (0)), f p (x p (n), u* p 2 (0))) > 8\ holds. Now we 
have to consider two cases: If x p (n + 1) = x x p f , then we can use the fact that the 

deviation d x {x p (\,x p {n)),x p p (l,x p (h))) > 5\ will occur again for some h > n 
due to the assumptions of the theorem. If x p (n + 1) ^ x r p f , we immediately obtain 
the existence of a 62 > such that \\x p (n + 1)11^ > 62 holds. In either case, we 
obtain that there exists a time index h> n such that Hx^n)!]^ > 8 = min((5i/2, 6 2 ) 
holds. 

Now suppose there exists a function e 7C£ such that (fl4l) holds for all n e No. 
Due to the X-property fi in its second argument, we have that for each e > there 
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exists an e N such that ||jc(n)||,.,ef < s for all n > h. Now we choose s < 6 and 
h e No accordingly. Since Lemma [T71 holds for all n e No, we can conclude that 
for h > n > n = h the inequality ||x(ft)|Lief > ||x p (n)|Lref > 6 > e holds. This 
contradicts the existence of a function f3 e 'KL such that (jl4l) holds for all n e N . 
Last, since the maps n and were chosen arbitrarily, the argumentation holds for 
all choices of n and which completes the proof. □ 

Remark 19. Condition I p (n, 1) + in Theorem [T8l is required since from q e 

it ^ it 2 
I p (n) we can only conclude that x p p (k n , x p (n)) and x p (k n , x p (rij) differ for some 

k n > 0. According to the NMPC algorithm, only the first control element is 
implemented and we may face the situation that again x p (k n+ i,x p (n +1)) and 

u' 2 

Xp (k n+ i,x p (n + 1)) differ for some k n+i > k n . Now if k n > holds for all n e N , 
then system p may be asymptotically stable. 

Turning from necessary to sufficient conditions we like to stress that the con- 
verse of Theorem [T8ldoes not hold, not even in the special case that the conditions 
of Lemma [T6l hold for all n > n with n e N . This conclusion is due to the fact 
that even if !P 2 = we can only guarantee that a control which minimizes © for 
all systems pef can be computed without having to consider any other system 
q e V \ {p}, but not whether all systems are actually stable. 

Theorem 20. Suppose that for given maps IX : 2 P — * 2 P we have that for a 
given initial value xq e X the set of admissible controls \] p p ' a (n, x p (n), I p (n)) is 
not empty for all p 6 V and all n 6 No. Suppose furthermore that there exist 
a\, a^, u\ e "7Co, y e 'TCoo and a > such that inequalities (fT6l) and (TTTT) hold for 
all n 6 No- Then there exists a function fj e < K£ which only depends on a, y and 
all a\, «2, p eP, such that (PT4l) holds for all n e N . 

Moreover, there exists an n e No such that for each n >nwe either have that 
Pi ±®,i>2 holds with I p (n, 1) = Qfor all p e P t or P 2 = 0. 

Proof. Using x e X, Up P ' a (n,x p (n),I p (n)) ± for all p e P and all n e N and 
Theorem [T3l we obtain that the closed loop solution x(n) = {x\{n) T , . . . , x P (n) T ) T 
exists for all n e No and satisfies x(n) e X for all n 6 No. Now, since inequalities 
(fT6l) and (TT71) hold for all n e No, the existence of ft e %£ follows directly from 
Proposition [TOl To show the existence of n e N such that for « > «we either 
have that P 2 = or P t ?Q),i>2 holds with I p (n, 1) = for all p € P u suppose 
that Pi + 0, i > 2 holds for some n > n and all n e N with I p (n, 1) + for 
some p e Pi. Then, using Theorem [T8l and the existence of fi e %£ we obtain a 
contradiction showing the assertion. □ 
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Remark 21. While the stability result of Theorem [2(JJis given for the NMPC case 
without stabilizing terminal constraints or terminal costs, the only critical compo- 
nent in the proof of this theorem is the condition that Up P ' ad (n, x p (n), I p (n)) £ 
which guarantees that the closed loop solution jc(-) = (xi(-) T , • • • ,*p(-) T ) T exists 
and satisfies the state constraints. Hence, if instead of the existence conditions of 
a\ , a p 2 , a\ e TCo and a > such that inequalities (fT6l) and ( fTTT ) hold we impose 
other stability conditions - e.g., the terminal constraint condition given in Keerthi 
and Gilbert [15] or the terminal costs from Chen and Allgower Q3J] - then the same 
proof can be used to guarantee asymptotic stability of the closed loop. 



5. Conclusion 

We presented a generalized stability result for NMPC controllers without sta- 
bilizing terminal constraints or terminal costs. Moreover, we described an algo- 
rithm which allows us to generate a hierarchy of such controllers in a distributed 
non cooperative setting. Using only abstract priority and testing maps, we have 
shown necessary as well as sufficient conditions for stability of the closed loop. 

Future research concerning the algorithm will certainly deal with the question 
how the priority and testing maps should be chosen to minimize the number of 
priority lists or to maximize the number of controllers that can be run in parallel. 
From the stability side an indeep analysis is required to apriori guarantee condition 
(fTTT ). The availability of such a condition would then allow us to apriori guarantee 
Algorithm [12] to asymptotically stabilize the system. One idea in this direction 
is outlined in Grime and Worthmann [|12l Section 7] and suggests the use of ISS 
small gain theorems to treat this problem. 
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